Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.96%. Comparing base (291a711) to head (77bc47d).
⚠️ Report is 13 commits behind head on master.

@@            Coverage Diff             @@
##           master    #7318      +/-   ##
==========================================
- Coverage   98.97%   98.96%   -0.02%     
==========================================
  Files          87       87              
  Lines       16733    16734       +1     
==========================================
- Hits        16561    16560       -1     
- Misses        172      174       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

the github action R-CMD-check / macos-15 is failing with the same error as reported in #6622

 ** testing if installed package can be loaded from temporary location
Error: Error: package or namespace load failed for ‘data.table’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/private/var/folders/sw/lqy3v8g55m76fhwckg439jjm0000gn/T/RtmpzQ41MM/Rinstfe91e48ef29/00LOCK-data.table/00new/data.table/libs/data_table.so':
  dlopen(/private/var/folders/sw/lqy3v8g55m76fhwckg439jjm0000gn/T/RtmpzQ41MM/Rinstfe91e48ef29/00LOCK-data.table/00new/data.table/libs/data_table.so, 0x0006): symbol not found in flat namespace '___kmpc_dispatch_deinit'
Error: Error: loading failed

Installing a custom OpenMP runtime seems to help, but this is somewhat fragile:

• The PKG_* environment variables are set globally for all source packages. We get most of our dependencies as binary packages from CRAN, so it only affects data.table. If there was a standard Make macro like SHLIB_OPENMP_LIBS documented in WRE, we would set it in ~/.R/Makevars when setting up the custom OpenMP runtime so that only packages that need OpenMP would be affected. Unfortunately, there's only SHLIB_OPENMP_CFLAGS/SHLIB_OPENMP_FFLAGS/SHLIB_OPENMP_CXXFLAGS, where the same flags are supposed to be passed to both the compiler and the linker (so SHLIB_OPENMP_LIBS would be counter-productive); only macOS needs different flags.
• Strictly speaking, this setup is not compatible with CRAN binaries. If we try to run a CRAN-built package that uses OpenMP together with our own source build of data.table, it will probably fail. But installing from source takes time and effort, and data.table is the only user of OpenMP during R CMD check, so it works.

great please add NEWS item

What would you suggest for the text of the news item when the change is mostly not user-visible? "The package is now also tested with Apple clang 17"?

sure that is fine.
I would disagree about "mostly not user-visible" -- easy installation is a really important user-visible feature.

Should the news item say that the compilation works with homebrew clang-17? (which is apparently different from xcode clang)

I have a response actions/runner-images#13012 (comment) from the github actions macos-15 image dev, who says: Clang is available as a part of Xcode and separately as an independent application instance installed from the Homebrew. Your #6622 referring to clang-17 which is not a part of our default config by the way. That means that one of your steps changes the default setup somehow.

to finalize this pr, we should revert the deletions I made in the R CMD check CI config.
But I would suggest keeping the macos-15 build that I added (and changing it to macos-latest), is that OK?

easy installation is a really important user-visible feature

Then we might want to add a configure test in addition to the current CI fixes. When running macOS and $(R CMD config CC) --version says Apple clang 17, check whether /usr/local/lib/libomp.dylib exists. If it does exist, use it instead of -lomp. If not, suggest installing the OpenMP runtime from https://mac.r-project.org/openmp and try using -lomp as before.

The risk is breaking installation from source on systems with Apple clang 17 where -lomp works but /usr/local/lib/libomp.dylib is broken. This sounds weird but not impossible in theory. Another risk is breaking installation on CRAN builders in the future, when they update to Xcode ≥16.0 and a new version of bundled libomp (so that -lomp would be sufficient).

Your #6622 referring to clang-17 which is not a part of our default config by the way.

This is technically correct (there's probably no binary named clang-17 in the macos-15 image), but doesn't contradict what's stated in #6622: the C compiler we're having problems with is Apple clang version 17 from Xcode 16.4, default in macos-15.

I found a test that distinguishes between working and broken OpenMP runtime situations on macOS: it's #pragma omp schedule(dynamic). Without the correct OpenMP runtime on Xcode 16.4, the resulting shared library fails to load due to missing ___kmpc_dispatch_deinit. This can be used in the configure test (à la #6642 but less intrusive), letting us be more confident about setting PKG_LIBS=/usr/local/lib/libomp.dylib for Apple clang 17. Give me some more time and we'll have a proper fix and a NEWS item here.

wow excellent job thanks Ivan!

Could you also add a runner with macos-15-intel to the matrix? That way we also test on x86_64, see https://github.blog/changelog/2025-09-19-github-actions-macos-13-runner-image-is-closing-down/

before this pr there were no macos github actions checks, so adding 3 seems like an unusually large move.

I would propose to add either 1 macos-15 or 2 macos (macos-15 for arm and macos-15-intel for x86_64).

It turns out that our caching scheme was in conflict with having macOS 15 running on both Intel and ARM.

Coincidentally, this enables OpenMP support on FreeBSD. Previously, the configure script only tried it for macOS and Linux; now it's tested everywhere, with extra cases on macOS.

Do we want to fail CI if data.table installs without OpenMP support on systems where it should be supported?

Would it be feasible for someone to test this with a Homebrew installation of R?

Do we want to fail CI if data.table installs without OpenMP support on systems where it should be supported?

No, I think it would be better to install single core. Anyway we dont see large speedups when adding cores, for many common operations https://github.com/Anirban166/data.table.threads

FYI, data.table builds on r-universe are broken because of this: http://cran.dev/data.table/buildlog

@jeroen, is there a way to get a CRAN-compatible toolchain on the macos-15 runner? Otherwise macOS builds produced on R-Universe using this runner should (and will, after this PR) have OpenMP disabled: the new compiler requires a new OpenMP runtime, but the new OpenMP runtime is incompatible with the one shipped with base R (and presumably used by other CRAN binaries).

I'm not sure. Could you ask this on the r-sig-mac mailing list? I'll try to update the headers to the latest from https://mac.r-project.org/openmp/ but IIUC they need to ship a newer openmp dylib with base R?

OK I found a workaround for GitHub actions:

The MacOS 15 GHA runners (both intel and arm64) also have some older versions of Xcode installed, see: https://github.com/actions/runner-images/blob/main/images/macos/macos-15-arm64-Readme.md

If we look at https://gist.github.com/yamaya/2924292, the last version of xcode that had clang 16 was Xcode_16.2.app. We can activate that using the following command:

sudo xcode-select -s /Applications/Xcode_16.2.app

After which the current master branch of data.table was able to build on MacOS 15 with OMP.

Of course this doesn't solve the problem but hopefully buys us some time until R for MacOS updates their libomp.dylib.

Dear @barracuda156, @kevinushey:

This rewrite of the configure script touches the use-cases (#6034, #6409) that I cannot easily test. Could you please help testing the new code? I'm worried about macOS configurations that aren't Xcode 14.x + the OpenMP runtime bundled with R or latest Xcode + the OpenMP runtime from https://mac.r-project.org/openmp.

The test should succeed if all exit codes are zero and the final command shows that some _OPENMP version is detected:

R CMD build .
mkdir -p data.table.Rcheck
R CMD INSTALL -l data.table.Rcheck data.table_1.17.99.tar.gz
R_LIBS_USER=data.table.Rcheck:%U R -q -s -e 'data.table::getDTthreads(TRUE)'

In particular, I'm interested in the configure script output shown by R CMD INSTALL. I think that at least some of the special cases will now be covered by our use of $(SHLIB_OPENMP_CFLAGS), but I'm less sure about Homebrew (and the many other possible configurations possible on macOS).

The libomp.dylib version that is included with R for MacOS has been updated in R-4.6-arm64 build 2025-12-14 r89168 to a newer version, which should be compatible with xcode 16.3 and up as it includes the ___kmpc_dispatch_deinit symbol.

For the R version 4.5 and lower, as well as all x86_64 versions the libomp.dylib does not get updated, so it is still needed to compile packages with xcode 16.2 or lower.

FYI users can completely avoid all problems by installing prebuilt binaries from https://rdatatable.r-universe.dev/data.table

Manually tested on a 10.15 Hackintosh with Clang 12 from Xcode CLT:

• CRAN build of R-4.2.3 + OpenMP runtime from https://mac.r-project.org/openmp/ + pkg-config and zlib stubs from https://mac.r-project.org/tools/: passes test for LIBS=-lomp
• MacPorts build of R-4.5.2:
  • +openmp: passes test for $(OPENMP_SHLIB_CFLAGS)
  • -openmp: passes test for -fopenmp (is this the right thing to do in a build of R where OpenMP was explicitly disabled?)
• Homebrew doesn't really support 10.15 and fails to download the nasm source tarball, I gave up
• GitLab CI used to pick up libomp from Homebrew, and now the jobs fail: 1 2. The runtime is alive and well, but the omp.h header is missing. According to Simon, this is deliberate: if a user wants to compile OpenMP-using packages from source, they really should follow the instructions at https://mac.r-project.org/tools/. I'll fix the configure test to check for the header too, but won't change GLCI configuration so that the Homebrew branch is still exercised.

@aitap MacPorts does not build R with Xcode clang. It picks specific LLVM clang (on most systems) and specific gcc (for gfortran on those, and as the primary compiler on legacy ones).
In both cases OpenMP is enabled: https://github.com/macports/macports-ports/blob/127c979274f93fd322bfe639871143caf3cfedd5/math/R/Portfile#L290

So unless you built R in a custom way with --enforce-variants -openmp, it is expected that OpenMP is detected.

P. S. R packages are stale in upstream MacPorts, but maintained here: https://github.com/macos-powerpc/R-ports
This repo is a submodule, which replaces R directory in ports. If you make a local overlay and git-clone it as R, add the path to sources.conf and run port sync, you can install R ports coherently with just port install.